Feat: Improve prompting for exploration efficiency#372
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Rate limit exceeded@ngoiyaeric has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 20 minutes and 52 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughReplaced hard-coded system prompts with a dynamic/default prompt in researcher, removed the Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant TaskMgr as Task Manager Agent
participant Researcher as Researcher Agent
participant Tools as Tools/GeoTools
Note over TaskMgr: Decision: "proceed" vs "inquire"
User->>TaskMgr: submit query
TaskMgr->>TaskMgr: evaluate clarity & geospatial needs
alt inquire needed
TaskMgr-->>User: ask clarifying questions ("inquire")
else proceed
TaskMgr->>Researcher: call researcher(dynamicSystemPrompt?, uiStream, streamText, messages, useSpecificModel?)
Researcher->>Tools: invoke tools (getTools({ uiStream, fullResponse }))
Tools-->>Researcher: tool results / errors
Researcher-->>TaskMgr: assembled assistant message (includes tool results or Error: Tool execution failed.)
TaskMgr-->>User: final response
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20–30 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
||||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||||
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
Overall, the new prompts are structurally clearer and better aligned with the stated goal of Exploration Efficiency, but there are a few issues worth addressing. In researcher.tsx, the pre-configured responses contain inconsistent/curly quotes and unclear string delimitation that could introduce ambiguity. In task-manager.tsx, the shift to efficiency-focused guidance omits the previous accuracy and uncertainty principles, which may unintentionally relax correctness guarantees. There is also a minor UX trade-off in how aggressively the task manager is instructed to "inquire" for broad queries, which may warrant slight tuning depending on product goals.
Additional notes (3)
- Maintainability |
lib/agents/researcher.tsx:32-41
The new prompt uses Markdown-style bolding (**...**) and list formatting. Depending on howsystemis consumed by the model, this formatting might or might not be interpreted as intended. If the previous system prompts were plain text and the model configuration isn't explicitly optimized for Markdown instructions, there’s a small risk that the extra formatting noise slightly dilutes the actual behavioral instructions.
Given this is a core behavioral prompt for the researcher agent, it’s worth double‑checking that this style is consistent with other agent prompts and with how this model is usually steered in your system.
- Maintainability |
lib/agents/researcher.tsx:57-60
The pre‑configured responses are now grouped under an efficiency section, but the actual wording appears unchanged, including some grammatical/quoting oddities and the description of "planet computer" as a "proprietary" system. If these responses are user‑visible marketing or product language, locking in this exact phrasing in a system prompt can make later copy changes harder and propagates any typos into all runs.
From a maintainability standpoint, this might be better centralized or referenced, rather than hard‑coded here, especially if other agents or surfaces need to answer these branded questions consistently.
- Readability |
lib/agents/task-manager.tsx:24-42
The newExploration Efficiency Principlesand examples are clear, but there is an implicit bias toward choosing "inquire" for broad queries like "Tell me about the new AI model." This is likely desired for token efficiency, but it may degrade UX for users who expect a reasonable, generic overview without being interrogated.
If the product requirement is to minimize back-and-forth while still avoiding obviously wasteful searches, you might want to tune this example or the language to clarify that "inquire" is preferred only when the breadth would make the subsequent exploration substantially inefficient, not merely because the topic is broad.
Summary of changes
Summary of Changes
- Updated the
researcheragent's default system prompt to emphasize Exploration Efficiency, including directives for tool usage, geospatial priority, search specificity, concise responses, and citations. - Simplified and restructured the geospatial tool usage documentation into a more compact "Tool Usage Guide" with example query categories.
- Added a Pre-configured Responses section for specific branded questions ("planet computer" and "QCX-Terra").
- Rewrote the
taskManageragent's system prompt to focus it as an efficient Task Manager, introducing Exploration Efficiency Principles, explicit guidance for choosing betweenproceedandinquire, and more concrete examples, especially for geospatial clarification. - Removed previous, more verbose data-accuracy/value principles from the
taskManagerprompt in favor of efficiency-oriented guidance.
| **Pre-configured Responses (for efficiency):** | ||
| When asked the following, respond accordingly: | ||
| - 'What is a planet computer?' answer with the following: '"A planet computer is a proprietary environment aware system that interoperates Climate forecasting, mapping and scheduling using cutting edge multi-agents to streamline automation and exploration on a planet' | ||
| - ‘What is QCX-Terra’ Respond with ‘QCX-Terra is a model garden of pixel level precision geospatial foundational models for efficient land prediction from satellite images’ |
There was a problem hiding this comment.
The new system prompt is significantly clearer and more structured, but there are a couple of minor issues that could affect robustness:
- Inconsistent quoting / smart quotes: The second pre-configured response uses a curly opening quote
‘instead of a standard ASCII'. Depending on how prompts are serialized or post-processed, this can introduce subtle bugs or inconsistencies in matching literal user utterances. - Unbalanced quotation in the first response: The answer string for "What is a planet computer?" starts with a double quote but does not clearly close it. That may confuse the model about where the quoted definition ends versus normal instruction text.
These are small, but cleaning them up will reduce ambiguity in the system message and make exact matching / prompt inspection easier.
Suggestion
Consider normalizing the quotes to standard ASCII and ensuring the definition strings are clearly delimited. For example:
When asked the following, respond accordingly:
What is a planet computer?→"A planet computer is a proprietary environment-aware system that interoperates climate forecasting, mapping, and scheduling using cutting-edge multi-agents to streamline automation and exploration on a planet."What is QCX-Terra?→"QCX-Terra is a model garden of pixel-level precision geospatial foundational models for efficient land prediction from satellite images."
Reply with "@CharlieHelps yes please" if you'd like me to add a commit that normalizes the quotes and clearly delimits these canned responses.
| **Examples of queries for 'geospatialQueryTool':** | ||
| Location Discovery: "Find coffee shops within walking distance of the Empire State Building" | ||
| Navigation & Travel: "Get driving directions from LAX to Hollywood with current traffic" | ||
| Visualization & Maps: "Create a map image showing the route from Golden Gate Bridge to Fisherman's Wharf" | ||
| Analysis & Planning: "Show me areas reachable within 30 minutes of downtown Portland by car" |
There was a problem hiding this comment.
The geospatial examples were significantly reduced compared to the original prompt (which covered discovery, navigation, visualization, analysis, and some more nuanced cases like travel time matrices and optimal routes). While the new list is more concise—which is good for efficiency—it may also remove useful priming for complex geospatial tasks the system is expected to handle.
If your user base frequently requests richer geospatial operations (e.g., multi‑point routing, travel time matrices, POI searches with constraints), the reduced examples might weaken the model’s tendency to use geospatialQueryTool for those patterns.
Suggestion
Consider re‑introducing one or two additional representative examples from the original prompt (e.g., a travel time matrix and an optimal route with multiple stops) while keeping the list concise. This preserves Exploration Efficiency but still gives the model good pattern coverage. Reply with "@CharlieHelps yes please" if you'd like me to add a commit that restores a couple of the higher‑value geospatial examples in a condensed form.
| system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent). | ||
|
|
||
| You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal: | ||
|
|
||
| **Exploration Efficiency Principles:** | ||
| - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches. | ||
| - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**. | ||
|
|
||
| **Options:** | ||
| 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately. | ||
| 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response. | ||
|
|
||
| **Inquiry Guidance (If "inquire" is chosen):** | ||
| - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool': | ||
| - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates. | ||
| - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking). | ||
| - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format"). | ||
|
|
||
| **Examples for Efficiency:** | ||
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search). | ||
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric). | ||
| - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location). | ||
| - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect). | ||
|
|
||
| Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user. | ||
|
|
There was a problem hiding this comment.
The new task manager prompt is much more focused on exploration efficiency and provides concrete guidance and examples, which is good. However, you removed all references to the previous accuracy, data-driven operations, and uncertainty transparency principles.
Given this agent decides whether to gather more info or proceed, dropping those principles entirely could implicitly relax standards around avoiding speculation and handling uncertainty, which may have been an intentional design constraint before.
If accuracy and non-speculation remain important system-level requirements, consider reincorporating a concise version of those principles, so the task manager does not favor efficiency at the expense of correctness.
Suggestion
You could append a short "Accuracy & Uncertainty" section to this prompt, preserving the spirit of the previous guidance without the verbosity, for example:
Accuracy & Uncertainty (Non-Negotiable):
- Base decisions on verifiable information when available.
- Avoid speculative assumptions; if critical information is missing, prefer "inquire".
- When uncertainty is high, prioritize gathering clarifying details over proceeding with a guess.
Reply with "@CharlieHelps yes please" if you'd like me to add a commit that reintroduces a compact accuracy/uncertainty block into this system prompt.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
lib/agents/task-manager.tsx (1)
8-16: Consider the implications of bypassing task manager for image inputs.The logic automatically proceeds when an image is present, skipping any potential need for clarification. This could be problematic if the user's query accompanying the image is still vague or ambiguous (e.g., "What's in this image?" without specifying what aspect they're interested in, or geospatial queries with images that lack location context).
Consider whether certain image-based queries might still benefit from the inquiry flow, or at minimum, document this design decision and its trade-offs.
lib/agents/researcher.tsx (1)
70-75: Fix inconsistent indentation.The indentation in this block is inconsistent, mixing different levels that don't align with the surrounding code structure. Lines 71-74 appear to be incorrectly indented relative to line 70.
Apply this diff to fix the indentation:
tools: getTools({ - uiStream, - fullResponse, - // mcp // mcp parameter is no longer passed to getTools - }) + uiStream, + fullResponse, + // mcp // mcp parameter is no longer passed to getTools + }) })
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
lib/agents/researcher.tsx(1 hunks)lib/agents/task-manager.tsx(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
lib/agents/task-manager.tsx (1)
app/actions.tsx (1)
processEvents(287-419)
lib/agents/researcher.tsx (6)
lib/agents/writer.tsx (1)
writer(7-51)lib/agents/tools/geospatial.tsx (2)
mcpClient(247-260)params(220-336)components/settings/components/settings.tsx (2)
fetchPrompt(81-86)fetchPrompt(80-88)lib/actions/chat.ts (1)
getSystemPrompt(239-254)lib/agents/tools/index.tsx (1)
ToolProps(7-11)components/settings/components/system-prompt-form.tsx (1)
SystemPromptForm(9-36)
🔇 Additional comments (2)
lib/agents/researcher.tsx (2)
32-61: Well-structured exploration efficiency directives.The new system prompt provides clear, actionable guidance on tool usage with explicit prioritization of the geospatial tool for location-based queries. The pre-configured responses and examples improve efficiency and consistency.
14-21: Breaking API change successfully implemented across all call sites.Verification confirms that the function signature change has been properly handled. The single call site at
app/actions.tsx:327passes all parameters in the correct order:currentSystemPrompt(mapped to the newdynamicSystemPromptparameter),uiStream,streamText,messages, anduseSpecificAPI. Themcpparameter has been correctly removed from the call.
| system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent). | ||
|
|
||
| You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal: | ||
|
|
||
| **Exploration Efficiency Principles:** | ||
| - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches. | ||
| - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**. | ||
|
|
||
| **Options:** | ||
| 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately. | ||
| 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response. | ||
|
|
||
| **Inquiry Guidance (If "inquire" is chosen):** | ||
| - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool': | ||
| - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates. | ||
| - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking). | ||
| - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format"). | ||
|
|
||
| **Examples for Efficiency:** | ||
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search). | ||
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric). | ||
| - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location). | ||
| - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect). | ||
|
|
||
| Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user. | ||
|
|
There was a problem hiding this comment.
Fix inconsistent indentation: mixed tabs and spaces.
The system prompt uses inconsistent indentation with tabs starting at line 22 (and continuing through the block), while line 20 uses spaces. This can cause issues with code formatting tools and reduces readability.
Apply this diff to standardize indentation (using spaces):
- system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent).
-
- You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal:
-
- **Exploration Efficiency Principles:**
- - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches.
- - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**.
-
- **Options:**
- 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately.
- 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response.
-
- **Inquiry Guidance (If "inquire" is chosen):**
- - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool':
- - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates.
- - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking).
- - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format").
-
- **Examples for Efficiency:**
- - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search).
- - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric).
- - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location).
- - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect).
-
- Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user.
-
+ system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent).
+
+ You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal:
+
+ **Exploration Efficiency Principles:**
+ - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches.
+ - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**.
+
+ **Options:**
+ 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately.
+ 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response.
+
+ **Inquiry Guidance (If "inquire" is chosen):**
+ - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool':
+ - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates.
+ - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking).
+ - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format").
+
+ **Examples for Efficiency:**
+ - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search).
+ - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric).
+ - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location).
+ - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect).
+
+ Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user.
+
`,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent). | |
| You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal: | |
| **Exploration Efficiency Principles:** | |
| - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches. | |
| - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**. | |
| **Options:** | |
| 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately. | |
| 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response. | |
| **Inquiry Guidance (If "inquire" is chosen):** | |
| - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool': | |
| - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates. | |
| - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking). | |
| - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format"). | |
| **Examples for Efficiency:** | |
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search). | |
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric). | |
| - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location). | |
| - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect). | |
| Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user. | |
| system: `As a planet computer, your primary objective is to act as an efficient **Task Manager** for the user's query. Your goal is to minimize unnecessary steps and maximize the efficiency of the subsequent exploration phase (researcher agent). | |
| You must first analyze the user's input and determine the optimal course of action. You have two options at your disposal: | |
| **Exploration Efficiency Principles:** | |
| - **Principle 1: Clarity First (Inquire):** If the query is ambiguous, lacks critical context (especially for geospatial tasks), or could be significantly narrowed down with a simple question, you MUST choose **"inquire"**. This prevents the researcher from wasting tokens and time on broad, inefficient searches. | |
| - **Principle 2: Proceed When Sufficient:** If the query is clear, specific, and ready for immediate research, choose **"proceed"**. | |
| **Options:** | |
| 1. **"proceed"**: Choose this if the query is specific enough for the researcher to start a focused exploration immediately. | |
| 2. **"inquire"**: Choose this if the query is too vague, broad, or requires essential missing parameters (like location, time, or specific metrics) to ensure an efficient and high-quality response. | |
| **Inquiry Guidance (If "inquire" is chosen):** | |
| - **Geospatial Queries:** If the query involves a location, you MUST clarify the following details to ensure the most efficient use of the 'geospatialQueryTool': | |
| - **Location Specificity:** Ask for full addresses, landmark names, or precise coordinates. | |
| - **Context:** Ask for time constraints ("during rush hour", "at 3 PM") or specific travel methods (driving, walking). | |
| - **Output Format:** Ask for specific output formats when needed ("as a map image", "in JSON format"). | |
| **Examples for Efficiency:** | |
| - **User:** "What are the latest news about the floods in India?" -> **Action:** "proceed" (Clear, ready for web search). | |
| - **User:** "What's the warmest temperature in my area?" -> **Action:** "inquire" (Missing location and preferred metric). | |
| - **User:** "Show me the nearest park." -> **Action:** "inquire" (Missing current location). | |
| - **User:** "Tell me about the new AI model." -> **Action:** "inquire" (Too broad; ask for the model name or specific aspect). | |
| Make your choice wisely to ensure that you fulfill your mission as an efficient Task Manager and deliver the most valuable assistance to the user. | |
| `, |
🤖 Prompt for AI Agents
lib/agents/task-manager.tsx lines 20-45: the system prompt block mixes tabs and
spaces (tabs start at line ~22) causing inconsistent indentation; normalize the
block to use spaces only (convert all leading tabs within the prompt to spaces,
match existing indentation width used on line 20, and ensure aligned
bullet/paragraph indentation) so formatting tools and editors see a consistent
indentation style.
Refactor researcher function to improve system prompt handling and update tool usage guidelines.
|
@coderabbitai the system is not returning real time information from the current news and date |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 5676 🏁 Script executed: Length of output: 3438 🏁 Script executed: Length of output: 27 🏁 Script executed: Length of output: 27 🏁 Script executed: Length of output: 0 🏁 Script executed: Length of output: 1383 🏁 Script executed: Length of output: 5380 🏁 Script executed: Length of output: 172 🏁 Script executed: Length of output: 1025 🏁 Script executed: Length of output: 2545 🏁 Script executed: Length of output: 0 🏁 Script executed: Length of output: 346 @ngoiyaeric I've analyzed the code and identified the likely issue. The system does have access to the current date and the search tool for real-time information, but the new prompt may not be enforcing tool usage strongly enough for news and current events. The IssueLooking at the new
Compare:
Recommended FixUpdate the const getDefaultSystemPrompt = (date: string) => raw`
As a comprehensive AI assistant, your primary directive is **Exploration Efficiency**. You must use the provided tools judiciously to gather information and formulate a response.
Current date and time: ${date}.
**Exploration Efficiency Directives:**
1. **Tool First:** Always check if a tool can directly or partially answer the user's query. Use the most specific tool available.
2. **Geospatial Priority:** For any query involving locations, places, addresses, geographical features, finding businesses, distances, or directions → you **MUST** use the 'geospatialQueryTool'.
3. **Real-Time Data Priority:** For any query about current events, news, recent developments, today's date, weather, or time-sensitive information → you **MUST** use the 'search' tool. Never rely on training data for current information.
4. **Search Specificity:** When using the 'search' tool, formulate queries that are as specific as possible.
5. **Concise Response:** When tools are not needed AND the query doesn't require real-time data, provide direct, helpful answers based on your knowledge. Match the user's language.
6. **Citations:** Always cite source URLs when using information from tools.
### **Tool Usage Guidelines (Mandatory)**
#### **1. General Web Search (MANDATORY for Real-Time Data)**
- **Tool**: \`search\`
- **MUST use for**:
• Current events, breaking news, recent developments
• Today's date, current time, weather conditions
• Latest statistics, prices, versions, or trending topics
• Any information that changes frequently (stock prices, sports scores, etc.)
- **When to use**:
Any query requiring up-to-date factual information or general knowledge.
- **Do NOT use** \`retrieve\` for URLs discovered via search results.
**Examples that REQUIRE \`search\`:**
- "What's the latest news about AI?"
- "Current weather in London"
- "What happened today in sports?"
- "Latest version of React"
- "Recent floods in India"Additional CheckCan you verify that Would you like me to help you update the prompt with these changes, or would you prefer to test the API key first? 🏁 Script executed: Length of output: 27 @ngoiyaeric Found the root cause! 🎯 ❌ Missing API KeyThe How to FixStep 1: Get a Tavily API Key
Step 2: Configure the Environment VariableAdd the key to your # Tavily API Key retrieved here: https://app.tavily.com/home
TAVILY_API_KEY=your_actual_key_hereStep 3: Restart Your ApplicationAfter adding the key, restart your development server or deployment for the changes to take effect. Additional RecommendationEven with the API key configured, I'd still recommend strengthening the prompt to enforce tool usage for real-time queries (as I suggested above). The current prompt doesn't mandate using the search tool for news/current events the same way it mandates using the geospatial tool for location queries. Would you like me to help you create an updated version of the
|
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
lib/agents/researcher.tsx(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/agents/researcher.tsx (2)
lib/utils/index.ts (1)
getModel(19-62)lib/agents/tools/index.tsx (1)
getTools(14-39)
🔇 Additional comments (5)
lib/agents/researcher.tsx (5)
8-8: Clear alias for stablestreamTextAPIAliasing
streamTexttononexperimental_streamTextmakes the migration away from the experimental API explicit at the call site; no issues here.
15-75: Default system prompt cleanly encodes Exploration Efficiency behaviorThe new
getDefaultSystemPromptis well-structured, clearly prioritizes tool usage (especiallygeospatialQueryTool), and the pre-configured responses now have balanced, ASCII quotes. From a code and prompt-structure standpoint, this looks solid.
99-105: ConfirmmaxTokens: 4096is safe for all possible models returned bygetModel()Since
getModel()may resolve to different backends (xAI Grok, Bedrock, or OpenAI), a hard-codedmaxTokens: 4096could exceed some providers’ maximum output token limits or recommended caps. Please double-check this value against the actual models you use, or consider driving it from configuration per provider to avoid runtime errors or silent truncation.
112-133: Streaming and answer-section timing improvements look goodShowing the
answerSectionon the first non-emptytext-delta, and on the firsttool-resultwhen not using a specific model, should improve perceived latency while avoiding an empty response shell. The accumulation oftoolCalls/toolResponsesin parallel also keeps the streaming loop straightforward.
143-150: Conversation state updates correctly preserve tool contextPushing an
assistantmessage that includes the accumulated text plustoolCalls, and then a separatetoolmessage containingtoolResponseswhen present, aligns well with a tools-first chat loop and should keep subsequent turns correctly grounded in prior tool usage.
| case 'error': | ||
| hasError = true | ||
| fullResponse += `\nError occurred while executing the tool` | ||
| fullResponse += `\n\nError: Tool execution failed.` | ||
| break |
There was a problem hiding this comment.
Error message is not propagated to the streamed UI
In the 'error' case you append Error: Tool execution failed. to fullResponse but never call streamText.update(fullResponse), so the user-visible BotMessage will not reflect the error text even though hasError is set.
Recommend updating streamText here as well:
case 'error':
hasError = true
- fullResponse += `\n\nError: Tool execution failed.`
+ fullResponse += `\n\nError: Tool execution failed.`
+ streamText.update(fullResponse)
breakThis keeps the UI and the accumulated response string in sync when tools fail.
🤖 Prompt for AI Agents
In lib/agents/researcher.tsx around lines 136-139, the 'error' branch appends
"Error: Tool execution failed." to fullResponse and sets hasError but never
updates the streamed UI; call streamText.update(fullResponse) after appending
the error (before the break) so the BotMessage receives the updated error text
and the streamed UI stays in sync with the accumulated response.
Changed dynamicSystemPrompt from optional to required.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
lib/agents/researcher.tsx (2)
128-139: Surface tool errors in the streamed answer text, not only viahasErrorYou now set
hasErrorin two places:
- When a
tool-resultarrives withoutdelta.result.- When an
'error'delta is emitted, appendingError: Tool execution failed.tofullResponse.But in neither case is the updated
fullResponsepushed to the user-visible stream. As a result, theBotMessagewill typically never show the error text; only the returnedhasErrorflag is set, which is easy for the UI to ignore.This is essentially the same issue previously flagged on the
'error'branch, and it now also applies to the new!delta.resultpath.Consider updating
streamTextwhenever you append error text so the user actually sees it:case 'tool-result': - if (!useSpecificModel && toolResponses.length === 0 && delta.result) { - uiStream.append(answerSection) - } - if (!delta.result) hasError = true + if (!useSpecificModel && toolResponses.length === 0 && delta.result) { + uiStream.append(answerSection) + } + if (!delta.result) { + hasError = true + fullResponse += `\n\nError: Tool returned no result.` + streamText.update(fullResponse) + } toolResponses.push(delta) break case 'error': hasError = true - fullResponse += `\n\nError: Tool execution failed.` + fullResponse += `\n\nError: Tool execution failed.` + streamText.update(fullResponse) breakThis keeps the streamed UI,
fullResponse, andhasErroraligned so tool failures are visible to end users.
71-74: Normalize pre‑configured questions for clearer matchingThe pre‑configured responses now use plain ASCII quotes, which is good. One small polish item:
- The second question string omits the trailing
?and both questions could be formatted consistently as literal prompts.To make matching more robust and easier to inspect, consider:
-**Pre-configured Responses:** -- "What is a planet computer?" → "A planet computer is a proprietary environment aware system that interoperates Climate forecasting, mapping and scheduling using cutting edge multi-agents to streamline automation and exploration on a planet" -- "What is QCX-Terra" → "QCX-Terra is a model garden of pixel level precision geospatial foundational models for efficient land prediction from satellite images" +**Pre-configured Responses:** +- `What is a planet computer?` → "A planet computer is a proprietary environment aware system that interoperates climate forecasting, mapping, and scheduling using cutting-edge multi-agents to streamline automation and exploration on a planet." +- `What is QCX-Terra?` → "QCX-Terra is a model garden of pixel-level-precision geospatial foundational models for efficient land prediction from satellite images."(You can keep or drop the grammar tweaks as you prefer; the key part is consistent, clearly delimited question strings.)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
lib/agents/researcher.tsx(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/agents/researcher.tsx (3)
components/message.tsx (1)
BotMessage(11-31)lib/utils/index.ts (1)
getModel(19-62)lib/agents/tools/index.tsx (2)
getTools(14-39)ToolProps(7-11)
🔇 Additional comments (2)
lib/agents/researcher.tsx (2)
8-9: Alias import avoids name clash withstreamTextparamRenaming the imported
streamTexttononexperimental_streamTextcleanly avoids shadowing thestreamTextfunction parameter and keeps call sites unambiguous. No issues here.
76-105: Dynamic system prompt handling and stream setup look solidMaking
dynamicSystemPromptoptional and falling back togetDefaultSystemPrompt(currentDate)only when it’s missing or all‑whitespace aligns the type with runtime behavior and simplifies callers. Thenonexperimental_streamTextinvocation withgetModel()as theLanguageModel, the increasedmaxTokens: 4096, and the wiring oftools: getTools({ uiStream, fullResponse })also look consistent with the surrounding code and the updated tools API.
| const getDefaultSystemPrompt = (date: string) => raw` | ||
| As a comprehensive AI assistant, your primary directive is **Exploration Efficiency**. You must use the provided tools judiciously to gather information and formulate a response. | ||
|
|
||
| Current date and time: ${date}. | ||
|
|
||
| **Exploration Efficiency Directives:** | ||
| 1. **Tool First:** Always check if a tool can directly or partially answer the user's query. Use the most specific tool available. | ||
| 2. **Geospatial Priority:** For any query involving locations, places, addresses, geographical features, finding businesses, distances, or directions → you **MUST** use the 'geospatialQueryTool'. | ||
| 3. **Search Specificity:** When using the 'search' tool, formulate queries that are as specific as possible. | ||
| 4. **Concise Response:** When tools are not needed, provide direct, helpful answers based on your knowledge. Match the user's language. | ||
| 5. **Citations:** Always cite source URLs when using information from tools. | ||
|
|
||
| ### **Tool Usage Guidelines (Mandatory)** | ||
|
|
||
| #### **1. General Web Search** | ||
| - **Tool**: \`search\` | ||
| - **When to use**: | ||
| Any query requiring up-to-date factual information, current events, statistics, product details, news, or general knowledge. | ||
| - **Do NOT use** \`retrieve\` for URLs discovered via search results. | ||
|
|
||
| #### **2. Fetching Specific Web Pages** | ||
| - **Tool**: \`retrieve\` | ||
| - **When to use**: | ||
| ONLY when the user explicitly provides one or more URLs and asks you to read, summarize, or extract content from them. | ||
| - **Never use** this tool proactively. | ||
|
|
||
| #### **3. Location, Geography, Navigation, and Mapping Queries** | ||
| - **Tool**: \`geospatialQueryTool\` → **MUST be used (no exceptions)** for: | ||
| • Finding places, businesses, "near me", distances, directions | ||
| • Travel times, routes, traffic, map generation | ||
| • Isochrones, travel-time matrices, multi-stop optimization | ||
|
|
||
| **Examples that trigger \`geospatialQueryTool\`:** | ||
| - “Coffee shops within 500 m of the Eiffel Tower” | ||
| - “Driving directions from LAX to Hollywood with current traffic” | ||
| - “Show me a map of museums in Paris” | ||
| - “How long to walk from Central Park to Times Square?” | ||
| - “Areas reachable in 30 minutes from downtown Portland” | ||
|
|
||
| **Behavior when using \`geospatialQueryTool\`:** | ||
| - Issue the tool call immediately | ||
| - In your final response: provide concise text only | ||
| - → NEVER say “the map will update” or “markers are being added” | ||
| - → Trust the system handles map rendering automatically | ||
|
|
||
| #### **Summary of Decision Flow** | ||
| 1. User gave explicit URLs? → \`retrieve\` | ||
| 2. Location/distance/direction/maps? → \`geospatialQueryTool\` (mandatory) | ||
| 3. Everything else needing external data? → \`search\` | ||
| 4. Otherwise → answer from knowledge | ||
|
|
||
| These rules override all previous instructions. | ||
|
|
There was a problem hiding this comment.
Tighten search vs. knowledge rules to reliably handle real‑time queries
The new default system prompt is much clearer and enforces geospatial tool usage well. However, the combination of:
- Directive 4 (“When tools are not needed, provide direct, helpful answers based on your knowledge”), and
- The “General Web Search” section that doesn’t explicitly require
searchfor current events/news,
still leaves room for the model to answer time‑sensitive questions from training data instead of calling search. That means the original “no real‑time info” symptom can still appear even once TAVILY_API_KEY is configured.
I’d recommend explicitly making search mandatory for time‑sensitive topics and narrowing when “tools are not needed,” for example:
-4. **Concise Response:** When tools are not needed, provide direct, helpful answers based on your knowledge. Match the user's language.
+4. **Concise Response:** Only answer directly from your knowledge when the question is clearly time-insensitive and does not require external or up-to-date data (e.g., math, stable scientific facts, definitions). Match the user's language.
-#### **1. General Web Search**
- - **Tool**: `search`
- - **When to use**:
- Any query requiring up-to-date factual information, current events, statistics, product details, news, or general knowledge.
+#### **1. General Web Search**
+- **Tool**: `search`
+- **When to use**:
+ For any query involving current events, news, changing prices/schedules, or other information that may have changed recently, you **MUST** call `search` at least once before answering.
+ For other queries, prefer `search` when the user explicitly asks you to look things up on the web (e.g., “search online”, “check the latest”, “look this up for me”).This keeps Exploration Efficiency while ensuring real‑time questions reliably hit the web instead of stale training data.
🤖 Prompt for AI Agents
In lib/agents/researcher.tsx around lines 18 to 70, tighten the system prompt so
time‑sensitive queries always trigger the search tool: change Directive 4 to
restrict “When tools are not needed” to only non‑time‑sensitive, general
knowledge (e.g., definitions, static facts) and add an explicit mandatory rule
in the General Web Search section that any time‑sensitive topics (current
events, news, launch dates, real‑time statistics, market prices, weather,
traffic, etc.) must use `search`; also update the decision flow to check “Is
this time‑sensitive?” → `search` before falling back to knowledge, and ensure
the geospatial mandatory rule remains unchanged.
Feat: Improve prompting for exploration efficiency
1 participant
User description
This pull request updates the system prompts for the
taskManagerandresearcheragents to explicitly prioritize Exploration Efficiency. \n\nChanges intaskManager.tsx:\n- The prompt is updated to act as an efficient Task Manager.\n- New Exploration Efficiency Principles are introduced to guide the decision betweenproceedandinquire.\n- Theinquireoption is now strongly tied to the Clarity First principle, especially for geospatial queries, to prevent the researcher from starting broad, token-intensive searches.\n\nChanges inresearcher.tsx:\n- The system prompt is updated with an Exploration Efficiency Directives section.\n- Directives emphasize Tool First, Geospatial Priority, and Search Specificity to ensure the most efficient use of resources and tools.PR Type
Enhancement
Description
Restructured system prompts to prioritize exploration efficiency
Added explicit "Exploration Efficiency Directives" to researcher agent
Introduced "Clarity First" principle to task manager for better query clarification
Reorganized tool usage guidance with clearer formatting and examples
Diagram Walkthrough
flowchart LR A["User Query"] --> B["Task Manager<br/>Clarity First Principle"] B --> C{"Query Clear?"} C -->|No| D["Inquire<br/>Gather Context"] C -->|Yes| E["Proceed"] D --> F["Researcher Agent<br/>Exploration Efficiency"] E --> F F --> G["Tool First:<br/>Select Optimal Tool"] G --> H["Geospatial Priority<br/>Location-based Queries"] H --> I["Search Specificity<br/>Focused Searches"] I --> J["Response with Citations"]File Walkthrough
researcher.tsx
Restructure researcher prompt for efficiency directiveslib/agents/researcher.tsx
Efficiency Directives" section
Specificity, Concise Response, and Citations
each category
task-manager.tsx
Refactor task manager prompt with clarity-first principlelib/agents/task-manager.tsx
Efficiency Principles"
ambiguous queries
clarification requirements
Summary by CodeRabbit
Enhancements
Changes
✏️ Tip: You can customize this high-level summary in your review settings.